KAFKA-20116: Forwad task-(end)-offset to task assignor (5/5)#22644
Merged
Conversation
mjsax
changed the title
mjsax
force-pushed
the
kafka-20116-forward-task-offsets
branch
from
d6573f2 to
cf629d5
Compare
dajac
reviewed
Jun 24, 2026
| // independently: a null list means "unchanged since the last heartbeat", so we retain the previously reported | ||
| // value for whichever of the two is null and only update when at least one is reported. | ||
| if (taskOffsets != null || taskEndOffsets != null) { | ||
| group.updateTaskOffsets(memberId, group.taskOffsets(memberId).update(taskOffsets, taskEndOffsets)); |
Member
There was a problem hiding this comment.
If they are transient, we may want to discuss whether using timeline data structures is the correct approach here. Those are usually backed by persisted records.
mjsax
force-pushed
the
kafka-20116-forward-task-offsets
branch
from
555347a to
c6380f3
Compare
mjsax
changed the title
mjsax
force-pushed
the
kafka-20116-forward-task-offsets
branch
from
6e34787 to
eb4345a
Compare
This PR adds in-memory state to store task-offset-sum and task-end-offset-sum, and wiring to forward both fields into the "streams" task assignor. We also ungate the warmup-tasks field on the heartbeat request handler. The wiring to forward warmup-tasks field existed already.
mjsax
force-pushed
the
kafka-20116-forward-task-offsets
branch
from
eb4345a to
2a069cc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds in-memory state to store task-offset-sum and
task-end-offset-sum, and wiring to forward both fields into the
"streams" task assignor.
We also ungate the warmup-tasks field on the heartbeat request handler.
The wiring to forward warmup-tasks field existed already.
Part of KIP-1071.
Reviewers: David Jacot david.jacot@gmail.com